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Abstract 


E-learning and online education offers important opportunities for educators as well as for students. Tra- 
ditional e-materials, as they are known today, do not allow the recognition of different parameters, such 
as: learning differences, prior knowledge, learning capabilities, learning environment, styles of learning, 
etc. Because e-materials are structured in such a way they cannot be successfully adapted for learners 
who consequently cannot control their own learning (Berge, 2002; Picciano, 2000; Saba, 2002). Such 
a result offers, among others, a highly anti-motivational effect. The preparation of modern e-materials 
therefore requires a thorough preparation in terms of content and design, which has to be (mostly) based 
on pedagogical and didactic theories. Modern e-materials, which can also be named educational e- 
materials, are usually accessible online (internet-based training (IBT), web-based training (WBT), online 
education, etc.), they enable and encourage self-learning, they are flexible, dynamic, interactive, use dif- 
ferent types of media, individualized and adapted to the user's needs. Mostly the latter characteristic will 
receive special attention in the following research. 

Key words: individualized e-learning, intelligent system, metadata. 


Introduction 


The nature of learning and creating environments for successful learning is currently in 
the center of trends in the field of education all around the world. Because of globalization most 
countries including Slovenia have set achieving high levels of knowledge and skills as a prior- 
ity task where special attention is given to more demanding forms of “competences for the 21“ 
century (Dumont, Istance, & Benavides, 2010; Sentoénik, Erculj, Rutar, AdleSi¢, & Zorman, 
2013). The key concern focuses on establishing that traditional approaches to education are not 
suited for fulfilling such demanding goals. 

Much has been said about possibilities which modern technologies offer in transforming 
education and training but only a handful of these statements are supported by research or even 
tested in thorough scientific researches (Lowe & Schnotz, 2008; Mayer, 2009; O’Neil & Perez, 
2003; Pytllik Zillig, Bodvarsson, & Bruning, 2005; Reiser & Dempsey, 2007; Rouet, Levonen, 
& Biardeau, 2001; Spector J. M., Merrill, Van Merrienboe, & Driscoll, 2008). Among other 
things, 1t is presumed that education will improve if student have access to laptops or tablet 
computers (for example: a personal digital assistant), to virtual reality games environment or 
by redirecting from teaching in person to virtual classroom. A certain part of this 1s true, but 
it needs to be emphasized that all the steps in this direction need to be carefully planned. In 
continuation the presented research gives special attention to research findings on learning with 
technology (science of learning), different ways technology can be used as support for learning 
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(science of teaching) and how on the bases of these successful educational environments can be 
created and what needs to be considered. 

Developing modern e-learning material is not a simple task, especially if this is to be 
adapted for an individual students. Some authors suggested that developing requires a thorough 
preparation in terms of content and design, which has to be (mostly) based on pedagogical and 
didactic theories (Bregar, Zagmajster, & Radovan, 2010), but others fields cannot be neglected. 
For making such individualized e-learning material at least three profiles of experts and exper- 
tise are needed (Ausubel, Novak, & Hanesian, 1978; Dolenc & Abersek, 2012): 

e Experts in the field of didactics, who are responsible for designing methodology for 
individualized e-learning materials. In order to make the material suitable for use in 
the learning environment, the following questions need to be answered: 


O how people perceive information and how information may be provided; 
O how people learn; 
O which methods and didactic principles can be applied; and to this suitable 


e-material needs to be adapted. 
e Content experts that have special professional subject knowledge and can answer 
the questions beneath: 


O which content is relevant to which knowledge 1n a given field; 
O how is this knowledge connected to the whole; 
O what is basic knowledge and what is an upgrade. 


e Experts in computer science or ICT to produce e-learning material to the standards 
and guidelines and to answer the following questions: 


O how computer programs are built and how on this basis e-learning materi- 
als can be built; 

O what are the technological capabilities of usage in education; 

O for which standards will e-learning materials be suitable for. 


Three steps are needed for creating qualitative and most of all individualized e-learning 
material. In the first step the learning content must be rearranged according to the individual’s 
needs and cut into smaller content pieces. In scientific and research publications numerous 
suggestions of cutting learning content on content assets and learning objects can be found 
(Andreev, Ganchev, & O’Droma, 2005; Bergstedt, Wiegreffe, Wittmann, & Moller, 2003; Lai, 
Liou, & Soc, 2007; Leal & Queiros, 2008). 

In the presented case study basic theory of programmed instruction is used, in which 
modular structure where the smallest piece of learning content represents a learning step is 
suggested (Dolenc & Abersek, 2012; Skinner, 1968; Strmicnik, 1978). In the second step, parts 
have to be sensibly combined into a whole. Material has to be created in such a way that it 
provides feedback to students (it guides them through the material), to teachers and creators of 
the material about students’ success at assimilating knowledge and other vital data; metadata. 
Metadata is data about data, they are small sets of descriptors that conveniently and efficiently 
describe, locate and work with other more extensive information sources (Pamela, Samuel, 
& Sissel, 2003). On the bases of metadata, analysis and evaluation of students’ educational 
achievements and the evaluation and optimization of the individualized e-learning material can 
be executed, which is the final and third step in creating and using the individualized e-learning 
material. The primary advantage of such a process and such type of individualized e-learning 
material is that it is flexible. It can be suited for the user and can be modified and supplemented 
regularly. 
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Theoretical Background 


Based on didactic guidelines of the programmed instruction (Frank, 1996; Skinner, 1968) 
for the greatest possible individualization, a hierarchical structure of the learning whole shown 
by Figure 1 was made in the first step. The learning whole is a very complex system. 
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Figure 1: Modular structure diagram of e-learning material. 


In presented research, attention is given only on building one of the blocks called “Gears”’. 
Building blocks are basic and essential elements that affect the quality of e-learning material. 
They consist of various learning steps and at the end they (must) have a summative assessment 
(Figure 1). With each building block certain knowledge standards are achieved which are re- 
quired by the specific subject curriculum (Fakin, Kocian¢éic¢, Hostnik, & Florjancic, 2011). The 
following things also have to be taken into consideration: 

e the didactic principle “step by step” approach which says that learning should start 

with a simple learning content and progress to a more complex one, 

e basic principles of Bloom cognitive taxonomy (Figure 1). 

In the second step, combined learning steps for building blocks “Gears” are made into an 
appropriate branched structure (Figure 2). 














Previous 
learning step 


Next learning 













Learning step 
Axles and shafts 


Legend: 

LC - learning content 

Q - formative assessment 

F - fedback with aditional content 

AE - learning step with aditional content 

HI - hyperlinks containing optional aditional content 
R - repeating 

Sr - skipped due to the repeating 


Figure 2: Flow chart of an individualized (for axles and shafts) learning step. 


Learning steps are not fragmented but connected to the previous and the following learn- 
ing steps. They are also adapted to individual needs, levels of knowledge and abilities of stu- 
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dents. Each learning step includes regular assessments with feedback that guides students from 
the beginning to the end. A formative assessment at the end is a part of each learning step. 

After it was used in the classroom, variables and metadata through the whole individual- 
ized e-learning material was collected. Variables and metadata therefore are component parts 
of the individualized e-learning material and after processing also become component parts for 
implementing analysis and evaluation of the individualized e-learning material and his users 
(Figure 3). 


developers: 


Experts in the field of didactics 
Experts in content 


Experts in computer science 


other institutions 
teacher (schools, goverment, 
universitys) 


teaching users 
learning a 


enviroment 








Figure 3: Use of metadata obtained in individualized e-learning material. 


When developing the individualized e-learning material metadata can be divided into 
two groups: 

e Internal, which is intended for the intelligent adaptation of the material to knowledge 

and style of the student, 1.e. individualization of the student’s educational pathway. 
This is the metadata, which the system in interaction with the individual gathers regu- 
larly, 1t processes it and creates new metadata, which is used to adapt the material and 
set up an adapted educational pathway for the user. Within this model it is possible to 
see the intelligence of the system, its capabilities of learning and adapting. 

e External, which 1s intended for the teachers, creators of the material and other institu- 
tions, 1.e. metadata which is intended for further analyses and evaluation. 

The use of metadata and variables does not end here. Appropriately processed metadata 
of the individual or a group of individuals can effectively be used by a teacher, who can on their 
bases: 

e Follow individual student’s knowledge and on the bases of it adjust the following les- 
son or repeat (complement, explains in a different manner) the same lesson if he or 
she, gathered from the data, determines that most students do not achieve the desired 
standards, 

e Offer additional learning assistance to a student or a group of students who do not 
achieve the desired standards, 

e During, or at the end of the school year assess the level of the student’s knowledge and 
if need be adjust teachers teaching methods, 

e Use the gathered metadata for self-analysis of a personal way of teaching and with the 
help of evaluation, if possible, improve methods of teaching and 
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e With regards to the experience gained from the classroom and gathered metadata, pro- 
pose certain changes and improvements to the creators of the individualized e-learning 
materials. 

Developers and creators of individualized e-learning materials can with the help of meta- 

data: 

e Adequately supplement segments of the individualized e-learning material that do not 
reach the desired results, 

e Add new didactic pathway in adequately evaluate the existing ones, 

e Enter new segments, which are based on different theories and test them directly in the 
learning environment, 

e Appropriately adjust and optimize the individualized e-learning material so it takes 
into account all levels of knowledge and gravitates towards better results. 

Other experts, can with the help of metadata adjust lesson plans, test new theories of 

teaching, implement research of performance on a national level and analyse and evaluate 
knowledge of the whole population also on an international level. 


Methodology of Research 


The object of this research is the development and evaluation of a pilot model of an intel- 
ligent individualized e-learning material for basic (lower secondary) school, in a subject course 
Science and Technology in the 8" grade. The topic of the individualized e-learning material is 
Gears, where student s are achieving the lowest results in national assessments (RIC, 2013). 
The pilot model was devised on the bases of didactic guidelines of programmed instructions 
and previously mentioned steps on how to develop such individualized e-learning material. 

With the help of an already made individualized e-material a pilot research study is being 
implemented, different metadata and variables are being collected. The pilot study 1s being car- 
ried out on a population of 100 students, where a traditional teaching model, with a teacher in 
the classroom is being directly compared to teaching with an individualized e-learning material, 
without a help of a teacher in the classroom (self-teaching). In the first testing stage of develop- 
ment methodology for individualized e-learning materials the prime topics of interest are: 

e the deviation between levels of knowledge in the individualized e-learning material 

(students self-learning) and traditional teaching 

e how the acquired metadata can be used for: improving materials, increasing knowledge 

of the individuals and analyzing and evaluating materials. 

With individualized e-learning material 250-300 pieces of metadata is gathered per stu- 
dent. Metadata is collected through whole of the individualized e-learning material. With every 
learning step and progression of the student different metadata is created within the individual- 
ized e-learning material, such as: starting and finishing time of a learning step, type of a learn- 
ing step, present level of a student’s knowledge on a learning step, decisions for progressing to 
a different learning step, selection of answers, type of additional help, type of questions, time 
need for answering, etc. At the same time all metadata generated by e-material itself is col- 
lected, such as: adjusted learning path, deviations from the initial level of knowledge, data on 
individual instructions given for improving results, etc. After every finished set all metadata is 
sent automatically by electronic mail (to developers) in a form of a chart (Figure 4). 
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Student name: miklavz2 Custom information 
Project name: Gonila (prvi del) 


Date completed: 6/11/2013 
Time completed: 08:38:46 
Total time spent in course: 62:49 


Custom variable 1: 7:36:36 
Custom variable 2: 1/1/1/1/1/2/1 
Custom variable 3: 1 


LARS SSS SSSR RRR RRR RRR RRR RRR RR RRR RRR SRR RRS SESS SEES ES Ee Custom variable 4: 7:59:06/0 


«x«x*«xe««* Formative check information«*«**** Custom variable 5: 7:59:09/0/8:02:10/0 


ROSS SSR RRR ERR RRR RRR RRR RS RRP PRP RP RPP PRPS. S| Custom variable 6: 1/0 


Interaction ID: 28338 Custom variable 7: 1/0/0/0 

Question Text: Izberi predstavnike strojnih elementov Custom variable 8: 1/1/0 

Interaction Type: Choice Custom variable 9: 0/0/0/0 

Current Date: 2013/06/11 Custom variable 10: 1/1/0/1/kotalIni 
Current Time: 08:34:26 Custom variable 11: 0/1/0/0/drsni 
Custom variable 12: 0/0 

Custom variable 13: 1/0/0 

Custom variable 14: 8:32:17/8:33:36 
Custom variable 15: 0/8:24:57/8:32:10 
Custom variable 16: 1/0/2/3/9/12 


Latency for question: 00:01:30 
Question was skipped: No 
Question answered correctly: true 
Users answer: A,D 

Correct answer: A,D 

Number of tries: 1 

Points for question: 1 

Question is a survey: false 





Figure 4: Example of data and metadata received by electronic mail. 


By collecting metadata in such a way it is possible to follow an individual throughout 
the whole individualized e-learning material and later assemble a complete learning path of the 
same individual. 


Participants and Sample Selection 


In the preliminary stage of testing the methodology for creating and evaluating individu- 
alized e-learning material, that contain the final pilot model of the individualized e-learning ma- 
terial only the first part of 1t was tested. The individualized e-learning material was tested in the 
9" grade in two randomly chosen elementary schools (suburban and urban), which have already 
studied the chosen subject matter in the 8" grade. 33 learners participated in the research. The 
individualized e-learning material was tested in a set of preparations for the national assessment 
in the field of Science and technology in the middle of the school year 2012/2013. 


Research Focus 


The research will in continuation focus solely on testing the methodology evaluated 
primarily from technological, programming perspective: 


e Adjusting the individualized e-learning material to the individual needs of the stu- 
dents, 

e Working on systems that collect and transform metadata, 

e Study of the suitability of didactic pathways and additional contents, 

e The way the individualized e-learning material works on different systems and brows- 
ers, 

e The stability of the system, which sends metadata, while simultaneously straining the 
server. 

In addition to technical perspective the following things were monitored: 

e How different individual pathways were generated, 

e The level of knowledge of students, who used the individualized e-learning material, 
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e How much time they spent learning with individualized e-learning materials, 
e The suitability of types of tasks and difficulty of the individualized e-learning mate- 
rial. 


Preliminary Research Results and Discussion 


With the preliminary research it was possible to successfully test technical functioning 
of individualized e-learning material. It adapted to students and the system for collecting and 
transforming metadata was functioning on all computers, systems and browsers. A problem 
was identified in the stability of the system which sends metadata. Because individualized e- 
learning material is independent and 1s not included in larger systems such as learning manage- 
ment system (LMS) or course management system (CMS)? not many suitable solutions exist 
for collecting and transferring data. The tests show that the so called user friendly solutions, 
where all metadata could be collected simultaneously and in a clear form, depend on browsers, 
types of systems and the processing power of computers that are used in schools. Because of 
this many errors occurred, which ended in a loss of some metadata. 

A certain amount of partial results were achieved concerning student’s achievements 
(level of knowledge, individual pathways, types of tasks, etc.), which confirmed the prior re- 
sults of national assessments (RIC, 2013). The national assessment show that the part of cur- 
ricula for Science and technology - Gears is the part with the poorest grades throughout entire 
testing. From the gathered metadata it was possible to get data on individual pathways, on the 
base of which analysis and evaluation of different bordering cases could be made, which could 
not be explained solely by the results of the national assessment. 

To state an example: In the first set there were three formative assessments and a summa- 
tive assessment. One of the student, who was a border case, has by studying with an individual- 
ized e-learning material in all formative assessments achieved maximum points, which means 
that he or she answered correctly to all questions and did not need any additional help so the 
material graded him/her, before the summative assessment, with the highest grade. But after the 
summative assessment the same student achieved a below average result (Figure 5). 


Formative Formative Formative SUMATIVE 


assessment assessment assessment ASSESSMENT 
Points: lof 1 Points: 2 of 2 Points: lof 1 Points: 1 of 13 





Figure 5: Formative and summative assessment results. 


If all metadata and the results of the formative assessments were not collected, a below 
average result would mean that the student falls under a category of the weaker students. With 
the results collected from the formative assessments it could be assumed that he/she had unau- 
thorized help in the first part or could not fully explain such a result. 

With an appropriate analysis and evaluation of the collected metadata it was established 
that the first part of the individualized e-learning material the student solved normally, without 
major complications, but at the summative assessment the time he or she used for answering 
the questions also called latency for question was 10 times slower than average latency of all 
students. For reading the questions and solving an individual task he/she in average used only 
3 seconds per task. Because the research was carried out in a controlled environment it was 
possible to acquire metadata on time intended for testing individualized e-learning material it 
can be claimed that the previously mentioned student used more time in the first part of the in- 
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dividualized e-learning material, and was running out of time for the last part of it. This means 
that in the desire to finish before the intended time for testing expired he/she only randomly 
skimmed through the summative assessment. By analyzing metadata of other participants it 
was established that students, who had an average or above average result in formative assess- 
ment, had similar problems related to momentary loss of motivation or lack of time. 


Conclusion 


Although the pilot research is still in progress it was possible, with the help of the meta- 
data from the preliminary research, to adequately adjust certain contents and didactic pathways 
of the individualized e-learning material, which proved to be inadequate, especially the system 
of transferring metadata. Thus it was decided that in order to solve the problem of transferring 
metadata, the metadata after every set in the pilot research needs to be transferred by electronic 
mail. Although this method requires more work for analysing and evaluating metadata it is be- 
cause of its simplicity more effective. 

Metadata from the preliminary research has benefited mostly all three types of experts 
who develop such individualized e-learning materials and who could on their bases adequately 
prepare all further sets of its. Acquired evaluated data was of the most benefit to the field con- 
nected to ICT and didactics. An extra building block was integrated into the material, in which 
the material independently analyses and evaluates the gathered metadata and seeks out border 
case students and in accordance with the received data provides instructions so they can try to 
achieve better results. In order to avoid time pressure and consequently poorer results the time 
needed for answering was prolonged. 
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